Backlight for touchless gesture detection

ABSTRACT

A device and method to detect a gesture performed by an object in touch-less communication with the device are described. The device includes two or more ambient light sensors arranged at respective first surface locations of the device, each of the two or more ambient light sensors sensing light intensity at the respective first surface location. The device also includes one or more processors to control one or more light sources at respective second surface locations of the device based on the light intensity sensed by the two or more ambient light sensors, calibrate the two or more ambient light sensors based on the one or more light sources, and detect the gesture based on the light intensity sensed by each of the two or more ambient light sensors.

BACKGROUND

Computers, mobile phones, tablets, and other computing and communicationdevices include a variety of input interfaces. For example, computersinclude keyboard and mouse input interfaces. Using a mouse, a user maynavigate to an area of the display and make a selection. For variousreasons, a mouse or equivalent input interface is not practical forhandheld devices such as tablets and mobile phones. One of those reasonsis that a handheld device is generally not used on a flat, stablesurface to accommodate mouse operation. Most tablets and many mobiledevices include capacitive touch sensors built under the screen that areused as a primary navigation and data input method. These touch sensorsrequire users to navigate the display with their finger, therebypotentially obstructing the screen as they try to make a selection andleaving finger prints and smudges on the screen over time.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of this disclosure, reference is nowmade to the following brief description, taken in connection with theaccompanying drawings and detailed description, wherein like referencenumerals represent like parts.

FIG. 1 shows a device including an exemplary arrangement of ambientlight sensors;

FIG. 2 depicts another view of the device shown in FIG. 1;

FIG. 3 shows a device including an exemplary arrangement of ambientlight sensors according to another embodiment;

FIG. 4 shows a device including an exemplary arrangement of ambientlight sensors according to yet another embodiment;

FIG. 5 is a block diagram of a system to process gestures;

FIG. 6 is a block diagram of a system to control the two or more ambientlight sensors;

FIG. 7 shows the process flow of a method of detecting a gesture;

FIG. 8 is a block diagram of an exemplary device that facilitatestouch-less gesture detection as described herein;

FIG. 9 shows an arrangement of backlight sources in an exemplary deviceaccording to an embodiment;

FIG. 10 is a block diagram of a system to control the backlight sourcesaccording to an embodiment; and

FIG. 11 is a process flow of an exemplary method of performingtouch-less gesture detection under any ambient lighting conditions.

DETAILED DESCRIPTION

It should be understood at the outset that although illustrativeimplementations of one or more embodiments of the present disclosure areprovided below, the disclosed systems and/or methods may be implementedusing any number of techniques, whether currently known or in existence.The disclosure should in no way be limited to the illustrativeimplementations, drawings, and techniques illustrated below, includingthe exemplary designs and implementations illustrated and describedherein, but may be modified within the scope of the appended claimsalong with their full scope of equivalents.

As noted above, many input interfaces are available for computation andcommunication devices. Embodiments of the system and method describedherein relate to touch-less gesture detection as a user interface.Additional embodiments relate to a light source and calibration based onthe light source to facilitate the gesture detection.

FIG. 1 shows a device 100 including an exemplary arrangement of ambientlight sensors 110. The device 100 may be any computation, communication,or data storage device such as a tablet, laptop computer, smart phone,music player, storage device, and the like. The view depicted by FIG. 1shows the screen 120 (e.g., glass or other transparent surface) of thedevice 100 on a surface of the body 125 that displays information to auser, which can be based on user selections or generated by the device100. Information generated by the device can include the status ofcommunication connections (mobile network, wifi connection(s), Bluetoothconnections, etc.), telephone call, or electronic messages or anycombination thereof. The screen 120 can act as the input/output (I/O)between the device 100 and the user. The exemplary device 100 shown inFIG. 1 has a screen 120 that occupies most of one surface of the device100. Other exemplary devices 100 may instead include a keyboard or othercomponents such that the relative size of the screen 120 to the size ofa surface of the device 100 is smaller than shown in FIG. 1 (see e.g.,FIG. 4). Three ambient light sensors (ALSs) 110 x, 110 y, 110 z aredisposed beneath the screen 120 in FIG. 1. Although the ALSs 110 areshown disposed beneath the screen 120 to protect from environmental andaccidental damage, the ALSs 110 receive the same intensity of ambientlight or at least sufficient ambient light to detect a change in ambientlight whether they are disposed above or below the screen 120, becausethe screen 120 is a transparent device element that allows ambient lightto pass through. The screen 120 includes a glass or polymer exteriorlayer that may filter or diffuse some light, e.g., certain ranges oflight wavelengths. Sufficient light for detection as described hereinpasses through the exterior layer of the screen 120. The ambient lightrefers to the available light (brightness and direction of light) in theenvironment in which the device 100 is being used. As such, the ALSs 110are passive devices. In an example, the ALSs 110 do not have and are notassociated with emitters on the device 100 to provide the light that isdetected by the ALSs 110. In a further example, the device 100 does notemit light for the purpose of gesture detection. Ambient light is, in anexample, the light present in the environment in which the device ispresent.

FIG. 2 depicts another view of the device 100 shown in FIG. 1. The viewshown by FIG. 2 includes a light source 210. This light source 210 maybe the sun, a lamp, or some combination of light sources that providethe available light in a given environment in which the device 100 isbeing used. If the device 100 is outside during the day, the sunprovides the ambient light, which is spread spectrum light. If thedevice is being used indoors with no exterior windows, the ambient lightis generated by indoor lighting systems, e.g. lamps, fluorescent bulbs,incandescent bulbs, LEDs, etc. The ambient light can also be acombination of natural light (e.g., sunlight) and artificial light(e.g., fluorescent light, incandescent light). Each ALS 110 outputs acurrent level corresponding with the measured light intensity 115 (seee.g., FIG. 5). An analog-to-digital converter may be used to derive adigital output from the ALSs 110. Each of the ALSs 110 may haveadjustable sensitivity (adjustable gain setting). Each ALS 110 may alsobe a spread spectrum sensor with a selectable range of operation amongtwo or more ranges (wavelength bands or ranges). The process entailed inthis selection is discussed further below with reference to FIG. 6. Thefull range of operation of each ALS 110 may be close to the wavelengthrange of visible light (400 nm to 700 nm). A typical commerciallyavailable ALS may detect ambient light in the wavelength range of 350 nmto 700 nm, for example. Because each ALS 110 measures the intensity ofthe available (ambient) light within its zone of reception (see e.g. 230y and 230 y′ defining a zone of reception for ALS 110 y and 230 z and230 z′ defining a zone of reception for ALS 110 z), the ALS 110 is apassive sensor that does not require a corresponding emitter ortransmitter. The zone of reception is typically cone-shaped with thecone dimensions being determined by an angle of half sensitivity. FIG. 2is a cross-sectional view of an exemplary zone of reception. Each ALS110 may measure light intensity 115 within its zone of reception in aphotometric unit (lux) to provide a measure of lumens per square-metersor in a radiometric unit (irradiance) to provide a measure of watts persquare-meters. In the embodiment shown by FIGS. 1 and 2, the three ALSs110 x, 110 y, 110 z are arranged in a triangular pattern. That is, atleast one ALS 110 is offset or not linearly aligned with at least twoother ALSs 110.

Through the inclusion of two or more ALSs 110 (e.g., three ALSs 110 x,110 y, 110 z), the device 100 shown in FIGS. 1 and 2 facilitatesdetection of a gesture by an object 240 that changes the light intensity115 (see e.g., FIG. 5) in the zone of detection of one or more of theALSs 110 due to movement of the object 240. Through the inclusion ofthree or more ALSs 110 with at least three of the three of more ALSs 110in a triangular pattern (see e.g., FIG. 1), movement of an object 240may be discerned in three dimensions. As is further detailed below, agesture is detected and identified based on the changes in lightintensity 115 measured by each of the ALSs 110 at different timeinstants or measurement cycles due to the movement of the object 240.That is, each of the ALSs 110 measures light intensity 115simultaneously with the other ALSs 110 at a given time instant or insequence with the other ALSs 110 for a measurement cycle, and thecomparison of light intensity 115 measurements for different timeinstants or measurement cycles is used to detect a gesture. For example,assuming that the ALSs 110 measure light intensity 115 simultaneously(or near-simultaneously), at the time instant illustrated by FIG. 2, theobject 240 is positioned such that the light intensity 115 detected byALS 110 z is affected but the light intensity 115 detected by ALSs 110 xand 110 y is unaffected by the object 240. Based on a direction ofmovement of the object 240, the light intensity 115 detected bydifferent ones of the ALSs 110 x, 110 y, 110 z may be affected atdifferent times instants by the position of the object 240. The object240 may be a hand, one or more fingers, a wand or anothernon-transparent item that partially or completely blocks the passage ofambient light so that its position may be detected based on the effecton measured light intensity 115.

A touch-free gesture may mimic a swipe, also known as a flick, which canbe a particular type of touch on a touch-sensitive display. The swipe orflick may begin at an origin point and continue to an end point, forexample, a concluding end of the gesture. A gesture may be identified byattributes or characteristics of the gesture as discussed further below.These attributes may include the origin point (of detection by an ALS110), the end point, the distance travelled by the object 240, theduration, the velocity, and the direction, for example. A gesture may belong or short in distance and/or duration. Two points of the gesture maybe utilized to determine a direction of the gesture. A gesture may alsoinclude a hover. A hover may be non-movement of the object 240 at alocation that is generally unchanged over a period of time.

In the arrangement of ALSs 110 shown in FIGS. 1 and 2, a minimumdistance may be required among the ALSs 110 x, 110 y, and 110 z (e.g.,distance 220 between ALSs 110 y and 110 z) in order to distinguish themovement of the object 240. This minimum distance may generally be onthe order of 2 centimeters (cm). More specifically, the minimum distancebetween ALSs 110 is based on an expected size of the object 240 as onefactor. For example, when an open hand is used as the object 240, agreater minimum distance may be required to distinguish a gesture thanwhen one finger is used as the object 240. This is because the open handwould cover all three ALSs 110 x, 110 y, 110 z at more time instantssuch that a movement of the open hand could only be distinguished whenthe object 240 is at an edge of the set of ALSs 110 x, 110 y, 110 z.According to one or more embodiments, the ALSs 110 may be positioned atthe corners or along the edges of the screen 120 and, thus, the screen120 size may determine the distance between the ALSs 110. When an openhand is anticipated to be the object 240 used to perform a gesture, aminimum distance between ALSs 110 of 3.5 cm may be used. The increaseddistance between ALSs 110 facilitates distinguishing the gesture (e.g.,direction, speed) more clearly, because all ALSs 110 will not be coveredby the open hand object 240 for the majority of the gesture movement.

Another distance that must be considered is the distance between theobject 240 and the ALS 110 (e.g., distance 250 between the object 240and ALS 110 z). First, as FIG. 2 makes clear, the object 240 must bebetween the light source 210 and the ALSs 110 in order to be detected byone or more of the ALSs 110 based on the effect of the object 240 onlight intensity 115 detected by one or more of the ALSs 110. While aminimum distance is generally not required between the object 240 and anALS 110 (i.e. the object 240 may almost touch the screen 120 surface),the object 240 may generally be 2-3 cm away from the screen 120 whileperforming the gesture. When the object 240 is too close to the ALSs 110(screen 120 surface), then some portion of the beginning or end of agesture may not be detected. This is due to the fact that the width ofthe zone of reception of the ALSs 110 (as shown in the cross-sectionaldepiction of FIG. 2 by 230 y and 230 y′ and by 230 z and 230 z′, forexample) is narrowest at the surface of the ALSs 110 and increases withincreased distance from the ALSs. Thus, as is clear from FIG. 2, anobject 240 that is closer in distance to an ALS 110 (screen 120 surface)must also be closer to a center of the ALS 110 (in the perpendiculardimension, along the screen 120) in order to enter the zone of receptionof the ALS 110. By hovering the object 240 above a given ALS 110 andmoving it farther away (reducing the object 240 effect and increasinglight intensity 115 measurement) or closer together (increasing theobject 240 effect and decreasing light intensity 115 measurement), agesture analogous to a mouse click may be made. Thus, double-click andtriple-click gestures may be added to available distinguishablegestures.

FIG. 3 shows a device 100 including an exemplary arrangement of ambientlight sensors 110 according to another embodiment. The exemplary device100 shown in FIG. 3 is similar to the device 100 shown in FIGS. 1 and 2in that the screen 120 occupies most of one surface of the device 100.The device 100 shown in FIG. 3 includes seven ALSs 110 a, 110 b, 110 c,110 d, 110 e, 110 f, 110 g arranged around the perimeter of the screen120. As shown in FIG. 3, ALS 110 a is offset from a common axial line111 of ALSs 110 b, 110 c, and 110 d and also a common axial line 111′ ofALSs 110 e, 110 f, and 110 g. In alternate embodiments, one or more ofthe ALSs 110 b, 110 c, and 110 d or the ALSs 110 e, 110 f, and 110 g maybe disposed such that they are not linearly aligned with other ALSs 110along 111 or 111′, respectively. For example, both ALS 110 c and ALS 110f may be disposed closer to the center of the screen 120 and, thus,offset from the axial line 111 common to ALSs 110 b and 110 d and theaxial line 111′ common to ALSs 110 e and 110 g, respectively. Increasingthe number of ALSs 110 increases the number of gestures that may bedetected by the device 100. For example, one waving gesture (movement ofthe object 240 from one side of the device 100 to the other) isillustrated by FIG. 3. Because of the number of ALSs 110 around theperimeter of the screen 120, other waving gestures, distinguishable fromthe waving gesture shown in FIG. 3, are also possible. The object 240may move from ALSs 110 d and 110 e to ALS 110 a, for example, or fromALS 110 d to ALS 110 g. It bears noting that, if the ALSs 110 wereclustered closer together and the object 240 is a hand, as shown in FIG.3, fewer distinguishable gestures are possible than when the ALSs 110are disposed, as shown.

FIG. 4 shows a device 100 including an exemplary arrangement of ambientlight sensors 110 according to yet another embodiment. Unlike theexemplary devices 100 shown in FIGS. 1-3, the device 100 shown in FIG. 4includes a keyboard or other component in the space 410 such that thescreen 120 occupies less of one surface of the device 100 relative tothe screen 120 shown in FIGS. 1-3. Three ALSs 110 m, 110 n, 110 o areshown near the perimeter of the screen 120. As noted above and shown inFIG. 1, the ALSs 110 m, 110 n, 110 o may be disposed closer together sothat the gestures made by the object 240 are more analogous to gesturesa user of a touchpad may make with a finger.

FIG. 5 is a block diagram of a system 500 to process gestures. Functionsperformed by the system 500 are discussed below with reference tospecific components. However, in alternate embodiments, the system 500may process gestures using one or more processors and one or more memorydevices that serve more than one of the functions discussed herein. Inaddition, the same processors and memory devices that process gesturesas discussed below may perform other functions within the device 100.For example, the processor to identify gestures may be one of severaldigital signal processors (DSPs 801, FIG. 8) generally available in asmart phone or tablet.

An input to the system 500 is the light intensity 115 measured from eachof the ALSs 110. The measurements are received by a data collectionengine 510, which includes both memory and processor functionalities. Asthe light intensity 115 measurement data is received from each of theALSs 110, the data collection engine 510 outputs a frame of data 520 foreach time instant. That is, each frame of data 520 includes the lightintensity 115 measurement for every ALS 110 at a given time instant.While each frame of data 520 may generally be discussed as including thelight intensity 115 measurement for each ALS 110 at an instant of time,the ALSs 110 may instead sample light intensity 115 in turn (rather thansimultaneously) such that a frame of data 520 includes light intensity115 measurements for a period of time for one cycle of the ALSs 110. Aprocessor functioning as a gesture identifier 530 receives each frame ofdata 520. The gesture identifier 530 may operate according to one ofseveral embodiments as discussed below.

In order to identify a movement of the object 240 as a particular(known) gesture, the gesture identifier 530 uses a comparison of lightintensity 115 measurements of the ALSs 110, as discussed below, alongwith a comparison with a gesture template 537 stored in a templatememory device 535. A dynamically adjusted minimum change in lightintensity 115 may be set based on expected noise and errors. That is, athreshold percentage of change in detected light intensity 115 may berequired before it is interpreted as a true variation in ambient light.Based on the light intensity 115 measurements among the ALSs 110 withina frame of data 520 (for a single time instant or measurement cycle),the gesture identifier 530 may ascertain a position of the object 240.For example, for a given frame of data 520, if the light intensity 115measurements of ALSs 110 d and 110 f are higher (by a defined threshold)than the light intensity 115 measurement output by ALS 110 e, then theobject 240 may be determined to be over the ALS 110 e and, thereby,blocking some of the light from the light source 210. Based on the lightintensity 115 measurements among two or more frames of data 520 (two ormore time instants or measurement cycles), the gesture identifier 530may ascertain characteristics of the (movement) gesture such as adirection of the movement, speed of the movement, and whether themovement is accelerating or decelerating. For example, if the lightintensity 115 measurements of ALSs 110 d and 110 f are higher (by adefined threshold) than the light intensity 115 measurement output byALS 110 e in one frame of data 520 and the light intensity 115measurement of ALS 110 e is higher (by a defined threshold) than thelight intensity 115 measurements output by ALSs 110 d and 110 f in thenext frame of data 520, the gesture identifier 530 may ascertain thatthe object 240 moved from a direction of the ALS 110 e toward adirection of the ALSs 110 d and 110 f. If the change in light intensity115 measurements occurred over several frames of data 520, then themovement of the object 240 may be ascertained as being relatively slowerthan if the change occurred over the course of one frame of data 240.Based on the ascertained characteristics of the gesture, the gestureidentifier 530 may identify the gesture among a set of known gesturesbased on the gesture template 537.

The gesture template 537 facilitates the association of a movement ofthe object 240 discerned by the gesture identifier 530 with a particularknown gesture. The gesture template 537 may be regarded as a sample ofideal light intensity 115 measurement data corresponding with each knowngesture. More specifically, the gesture template 537 may be regarded asproviding the ideal relative light intensity 115 among the ALSs 110 orframes of data 520 or both for a given known gesture. Thus, by comparingthe input light intensity 115 measurements (in the frames of data 520)or comparisons of light intensity measurements 115 with the idealmeasurements in the gesture template 537, the gesture identifier 530identifies the object 240 movement as a known gesture. Thisidentification of the gesture may be done by a process of elimination ofthe known gestures in the gesture template 537. Thus, the gestureidentifier 530 may identify the gesture using the gesture template 537,through a process of elimination of available known gestures, before theobject 240 movement is complete. In this case, the gesture identifier530 may continue to process frames of data 520 to verify the detectedgesture or, in alternate embodiments, the gesture identifier 530 maystop processing additional frames of data 520 after identifying thegesture and wait for a trigger signal 540 discussed below. Each of theALSs 110 may be programmable to provide 10, 20, 50, 10, 125, 15, 200 and250 samples of light intensity 115 (frames of data 520) a second. TheALS 110 scanning rate is a factor in determining the speed at which agesture may be made in order to be recognized. That is, when the ALSs110 are sampling at a rate of 10 light intensity 115 samples per second,the fastest identifiable gesture is much slower than the fastestidentifiable gesture that may be made when the ALSs 110 are sampling ata rate of 250 light intensity 115 samples per second. The ALSs 115sampling at a rate of 10 frames of data 520 per second (10 lightintensity 115 samples per second each) may translate to an object 240travelling 10 cm in 1.5 seconds in order to be recognized and processedproperly. The system 610 (FIG. 6) may dynamically calculate and adjustthe scanning rate of the ALSs 110.

Another input to the gesture identifier 530 is one of the gesturelibraries 555 stored in a gesture library storage 550. Each gesturelibrary 555 is associated with an application, and the gestureidentifier 530 selects the gesture library 555 associated with theapplication currently being executed by the device 100. A given gesturelibrary 555 associated with a given application may not include everyknown gesture in the gesture template 537. Thus, based on theapplication currently being executed by the device 100, the gestureidentifier 530 may narrow down the set of known gestures within thegesture template 537 to compare against the frames of data 520 output bythe data collection engine 510 in order to identify the gesture. Agesture library 555 indicates an action output 560 corresponding with aset of gestures. Thus, when the gesture identifier 530 identifies aknown gesture based on the movement of the object 240 and the gesturetemplate 537, and the gesture identifier 530 finds that known gestureamong the set of gestures in a gesture library 555 associated with theapplication currently being run by the device 100, then the gestureidentifier 530 outputs the corresponding action output 560 stemming fromthe object 240 movement. The action output 560 of the gesture identifier530 acts as a command to the application being executed. For example,when the application being executed is a document editing session, thegestures identified by the gesture identifier 530 may correspond withaction outputs 560 such as “next page” (wave down), “previous page”(wave up), “zoom in” (bringing fingers together), and “zoom out”(spreading fingers apart). If the device 100 is currently not executingany application or if the application currently being executed by thedevice 100 does not have a gesture library 555 associated with it, then,even if the gesture identifier 530 uses the gesture template 537 toidentify a known gesture based on the movement of the object 240, noaction is taken by the gesture identifier 530 based on identifying thegesture. That is, there is no action output 560 corresponding with theidentified gesture, because there is no gesture library 555 to look up.

According to one embodiment, the gesture identifier 530 may not use thegesture template 537 to identify a gesture when no application is beingexecuted by the device 100 or when an application without an associatedgesture library 555 is being executed by the device 100. According toanother embodiment, the gesture identifier 530 may not begin to processany frames of data 520 before receiving a trigger signal 540. Thetrigger signal 540 is detailed below with reference to FIG. 6. Accordingto another embodiment, the gesture identifier 530 may process an initialset of frames of data 520 and then not process another set of frames ofdata 520 needed to identify the gesture until the trigger signal 540 isreceived. For example, the gesture identifier 530 may process aparticular number of frames of data 520 or a number of frames of data520 representing a particular length of time (number of time instants)and then stop processing further frames of data 520 until the triggersignal 540 is received. According to yet another embodiment, the gestureidentifier 530 may continually process frames of data 520 as they areoutput from the data collection engine 510.

Regardless of the behavior of the gesture identifier 530 based on thetrigger signal 540, the lack of an associated gesture library 555, orthe lack of an application being executed at all, the data collectionengine 510 still outputs the frames of data 520. This is because thelight intensity 115 measurements may be used for background functionssuch as adjustment of the screen 120 backlighting, for example, based onthe detected ambient light, even if gesture detection is not to beperformed. Some of these background functions are detailed below withreference to FIG. 6.

FIG. 6 is a block diagram of a system 610 to control the two or moreambient light sensors 110. As noted with reference to FIG. 5, thefunctions described for the system 610 may be performed by one or moreprocessors and one or more memory devices, which may also perform otherfunctions within the device 100. The system 610 may be regarded as abackground processing system, because it may operate continuously todynamically control the ALSs 110. The system 610 receives the lightintensity 115 measurements output by the ALSs 110 to the data collectionengine 510 as frames of data 520. In alternate embodiments, the ALSs 110may directly output light intensity 115 measurements to the system 610as well as to the data collection engine 510. The system 610 may alsoreceive additional information 620. This additional information 620 mayindicate, for example, whether the device 100 is currently executing anapplication and, if so, which application the device 100 is currentlyexecuting.

Based on the light intensity 115 measurements (directly or in the formof frames of data 520) and the additional information 620, the system610 adjusts the sensitivity or wavelength band or range or both for eachALS 110. For example, based on the available light (measured ambientlight intensity 115), the system 610 may change the wavelength range forthe ALSs 110 via a control signal 630 from the system 610 to one or moreof the ALSs 110. The change (adjustment of wavelength range) may ensurethat the ALSs 110 are focused in the correct wavelength (frequency) bandfor the current conditions. As another example, based on a change inavailable light (e.g., based on switching a light on or off), the system610 may change the sensitivity of the ALSs 110. Any order of switchinglights produces a new range of change in light intensity 115 to whichthe ALSs 110 must adapt. For example, the range of change of lightintensity 115 to which the ALSs 110 are sensitive may be 50-250 lux. Ina darker environment (e.g., a conference room during a presentation) therange of change of light intensity 115 to which the ALSs 110 aresensitive may be 2-15 lux. The adjustment of the ALSs 110 through thecontrol signal 630 may be done continuously, periodically, or based on atrigger event such as, for example, a change in the application beingexecuted by the device 100. For example, sensitivity adjustment may bedone automatically once for every 5 frames of data 520. The system 610may also adjust the order and frequency of light intensity 115measurements by the ALSs 110. For example, based on additionalinformation 620 indicating that a particular application is beingexecuted by the device 100, the system 610 may send control signals 630to have the ALSs 110 collect light intensity 115 samples for each cycle(frame of data 520) in a particular order and with a particularfrequency.

In addition to controlling the ALSs 110, the system 610 may provide thetrigger signal 540 to the gesture identifier 530 (see FIG. 5). Becausethe system 610 monitors the light intensity 115 measurements in theframes of data 520 to fulfill the background functions described above,the system 610 may additionally identify trigger events that signal whengesture processing should be initiated by the gesture identifier 530 andoutput the trigger signal 540 accordingly. For example, the system 610may output a trigger signal 540 to the gesture identifier 530 when itreceives a frame of data 520 that indicates a change in light intensity115 measured by one or more ALSs 110. The change in light intensity 115measurement may indicate a start of a movement of an object 240 and,thus, the start of a gesture. In various embodiments, the change inmeasured light intensity 115 may be 10%+/−3% or higher before the system610 outputs a trigger signal 540. In an embodiment, the change inmeasured light intensity 115 may be 20%+/−5% or higher before the system610 outputs a trigger signal 540. In an embodiment, the change inmeasured light intensity may be 25%+/−5% or higher before the system 610outputs a trigger signal 540.

FIG. 7 shows the process flow of a method 700 of detecting a gestureaccording to embodiments discussed above. At block 710, arranging two ormore ALSs 110 under the screen 120 of a device 100 may be according tothe embodiments shown in FIGS. 1, 3, and 4 or in alternate arrangementsaccording to the guidelines discussed above. Obtaining light intensity115 measurements from the ALSs 110 (block 720) may be in photometric orradiometric units as discussed above. Obtaining (receiving) the lightintensity 115 measurements may also include dynamically controlling theALSs 110 with the system 610 to modify the wavelength range or spectralsensitivity of each ALS 110, for example. As discussed with reference toFIG. 6, the control by the system 610 may be based on light intensity115 measurements by the ALSs 110, for example. Determining what, if any,application is being executed by the device 100, at block 730, may bedone by the gesture identifier 530 and may be part of the additionalinformation 620 provided to the system 610. At block 740, the processincludes storing a gesture library 555 associated with each applicationthat may be operated using touch-less gestures in the gesture librarystorage 550. Selecting the gesture library 555 associated with theapplication being executed by the device 100 may be done by the gestureidentifier 530 at block 750. Block 750 may also include the gestureidentifier 530 determining that no gesture library 555 is applicablebecause the device 100 is not executing any application or is executingan application without an associated gesture library 555. At block 760,processing the light intensity 115 measurements and identifying agesture involves the data collection engine 510 outputting the frames ofdata 520 and the gesture identifier 530 using a comparison of lightintensity 115 measurements in addition to the gesture template 537.Block 760 may also include the system 610 sending a trigger signal 540to the gesture identifier 530 to begin or continue the gestureprocessing. Block 760 may further include the gesture identifier 530 notidentifying the gesture at all based on not having a gesture library 555available. At block 770, outputting an action signal 560 correspondingwith the gesture based on the gesture library 555 is done by the gestureidentifier 530 as detailed above.

FIG. 8 is a block diagram of an exemplary device 100 that facilitatestouch-less gesture detection as described in embodiments above. Whilevarious components of the device 100 are depicted, alternate embodimentsof the device 100 may include a subset of the components shown orinclude additional components not shown in FIG. 8. The device 100includes a DSP 801 and a memory 802. The DSP 801 and memory 802 mayprovide, in part or in whole, the functionality of the system 500 (FIG.5). As shown, the device 100 may further include an antenna andfront-end unit 803, a radio frequency (RF) transceiver 804, an analogbaseband processing unit 805, a microphone 806, an earpiece speaker 807,a headset port 808, a bus 809, such as a system bus or an input/output(I/O) interface bus, a removable memory card 810, a universal serial bus(USB) port 811, an alert 812, a keypad 813, a short range wirelesscommunication sub-system 814, a liquid crystal display (LCD) 815, whichmay include a touch sensitive surface, an LCD controller 816, acharge-coupled device (CCD) camera 817, a camera controller 818, and aglobal positioning system (GPS) sensor 819, and a power managementmodule 820 operably coupled to a power storage unit, such as a battery826. In various embodiments, the device 100 may include another kind ofdisplay that does not provide a touch sensitive screen. In oneembodiment, the DSP 801 communicates directly with the memory 802without passing through the input/output interface (“Bus”) 809.

In various embodiments, the DSP 801 or some other form of controller orcentral processing unit (CPU) operates to control the various componentsof the device 100 in accordance with embedded software or firmwarestored in memory 802 or stored in memory contained within the DSP 801itself. In addition to the embedded software or firmware, the DSP 801may execute other applications stored in the memory 802 or madeavailable via information media such as portable data storage media likethe removable memory card 810 or via wired or wireless networkcommunications. The application software may comprise a compiled set ofmachine-readable instructions that configure the DSP 801 to provide thedesired functionality, or the application software may be high-levelsoftware instructions to be processed by an interpreter or compiler toindirectly configure the DSP 801.

The antenna and front-end unit 803 may be provided to convert betweenwireless signals and electrical signals, enabling the device 100 to sendand receive information from a cellular network or some other availablewireless communications network or from a peer device 100. In anembodiment, the antenna and front-end unit 803 may include multipleantennas to support beam forming and/or multiple input multiple output(MIMO) operations. As is known to those skilled in the art, MIMOoperations may provide spatial diversity, which can be used to overcomedifficult channel conditions or to increase channel throughput.Likewise, the antenna and front-end unit 803 may include antenna tuningor impedance matching components, RF power amplifiers, or low noiseamplifiers.

In various embodiments, the RF transceiver 804 facilitates frequencyshifting, converting received RF signals to baseband and convertingbaseband transmit signals to RF. In some descriptions a radiotransceiver or RF transceiver may be understood to include other signalprocessing functionality such as modulation/demodulation,coding/decoding, interleaving/deinterleaving, spreading/despreading,inverse fast Fourier transforming (IFFT)/fast Fourier transforming(FFT), cyclic prefix appending/removal, and other signal processingfunctions. For the purposes of clarity, the description here separatesthe description of this signal processing from the RF and/or radio stageand conceptually allocates that signal processing to the analog basebandprocessing unit 805 or the DSP 801 or other central processing unit. Insome embodiments, the RF Transceiver 804, portions of the antenna andfront-end unit 803, and the analog base band processing unit 805 may becombined in one or more processing units and/or application specificintegrated circuits (ASICs).

Note that, in this diagram, the radio access technology (RAT) RAT1 andRAT2 transceivers 821, 822, the IXRF 823, the IRSL 824 and Multi-RATsubsystem 825 are operably coupled to the RF transceiver 804 and analogbaseband processing unit 805 and then also coupled to the antenna andfront-end unit 803 via the RF transceiver 804. As there may be multipleRAT transceivers, there will typically be multiple antennas or frontends 803 or RF transceivers 804, one for each RAT or band of operation.

The analog baseband processing unit 805 may provide various analogprocessing of inputs and outputs for the RF transceivers 804 and thespeech interfaces (806, 807, 808). For example, the analog basebandprocessing unit 805 receives inputs from the microphone 806 and theheadset 808 and provides outputs to the earpiece 807 and the headset808. To that end, the analog baseband processing unit 805 may have portsfor connecting to the built-in microphone 806 and the earpiece speaker807 that enable the device 100 to be used as a cell phone. The analogbaseband processing unit 805 may further include a port for connectingto a headset or other hands-free microphone and speaker configuration.The analog baseband processing unit 805 may provide digital-to-analogconversion in one signal direction and analog-to-digital conversion inthe opposing signal direction. In various embodiments, at least some ofthe functionality of the analog baseband processing unit 805 may beprovided by digital processing components, for example by the DSP 801 orby other central processing units.

The DSP 801 may perform modulation/demodulation, coding/decoding,interleaving/deinterleaving, spreading/despreading, inverse fast Fouriertransforming (IFFT)/fast Fourier transforming (FFT), cyclic prefixappending/removal, and other signal processing functions associated withwireless communications. In an embodiment, for example in a codedivision multiple access (CDMA) technology application, for atransmitter function the DSP 801 may perform modulation, coding,interleaving, and spreading, and for a receiver function the DSP 801 mayperform despreading, deinterleaving, decoding, and demodulation. Inanother embodiment, for example in an orthogonal frequency divisionmultiplex access (OFDMA) technology application, for the transmitterfunction the DSP 801 may perform modulation, coding, interleaving,inverse fast Fourier transforming, and cyclic prefix appending, and fora receiver function the DSP 801 may perform cyclic prefix removal, fastFourier transforming, deinterleaving, decoding, and demodulation. Inother wireless technology applications, yet other signal processingfunctions and combinations of signal processing functions may beperformed by the DSP 801.

The DSP 801 may communicate with a wireless network via the analogbaseband processing unit 805. In some embodiments, the communication mayprovide Internet connectivity, enabling a user to gain access to contenton the Internet and to send and receive e-mail or text messages. Theinput/output interface (“Bus”) 809 interconnects the DSP 801 and variousmemories and interfaces. The memory 802 and the removable memory card810 may provide software and data to configure the operation of the DSP801. Among the interfaces may be the USB interface 811 and the shortrange wireless communication sub-system 814. The USB interface 811 maybe used to charge the device 100 and may also enable the device 100 tofunction as a peripheral device to exchange information with a personalcomputer or other computer system. The short range wirelesscommunication sub-system 814 may include an infrared port, a Bluetoothinterface, an IEEE 802.11 compliant wireless interface, or any othershort range wireless communication sub-system, which may enable thedevice to communicate wirelessly with other nearby client nodes andaccess nodes. The short-range wireless communication sub-system 814 mayalso include suitable RF Transceiver, Antenna and Front End subsystems.

The input/output interface (“Bus”) 809 may further connect the DSP 801to the alert 812 that, when triggered, causes the device 100 to providea notice to the user, for example, by ringing, playing a melody, orvibrating. The alert 812 may serve as a mechanism for alerting the userto any of various events such as an incoming call, a new text message,and an appointment reminder by silently vibrating, or by playing aspecific pre-assigned melody for a particular caller.

The keypad 813 couples to the DSP 801 via the I/O interface (“Bus”) 809to provide one mechanism for the user to make selections, enterinformation, and otherwise provide input to the device 100. The keypad813 may be a full or reduced alphanumeric keyboard such as QWERTY,DVORAK, AZERTY and sequential types, or a traditional numeric keypadwith alphabet letters associated with a telephone keypad. The input keysmay likewise include a track wheel, track pad, an exit or escape key, atrackball, and other navigational or functional keys, which may beinwardly depressed to provide further input function. Another inputmechanism may be the LCD 815, which may include touch screen capabilityand also display text and/or graphics to the user. The LCD controller816 couples the DSP 801 to the LCD 815.

The CCD camera 817, if equipped, enables the device 100 to make digitalpictures. The DSP 801 communicates with the CCD camera 817 via thecamera controller 818. In another embodiment, a camera operatingaccording to a technology other than Charge Coupled Device cameras maybe employed. The GPS sensor 819 is coupled to the DSP 801 to decodeglobal positioning system signals or other navigational signals, therebyenabling the device 100 to determine its position. The GPS sensor 819may be coupled to an antenna and front end (not shown) suitable for itsband of operation. Various other peripherals may also be included toprovide additional functions, such as radio and television reception.

In various embodiments, device 100 comprises a first Radio AccessTechnology (RAT) transceiver 821 and a second RAT transceiver 822. Asshown in FIG. 8, and described in greater detail herein, the RATtransceivers ‘1’ 821 and ‘2’ 822 are in turn coupled to a multi-RATcommunications subsystem 825 by an Inter-RAT Supervisory Layer Module824. In turn, the multi-RAT communications subsystem 825 is operablycoupled to the Bus 809. Optionally, the respective radio protocol layersof the first Radio Access Technology (RAT) transceiver 821 and thesecond RAT transceiver 822 are operably coupled to one another throughan Inter-RAT eXchange Function (IRXF) Module 823.

As noted above in the discussion of the system 610, the sensitivity ofthe ALSs 110 may be adjusted based on the available light. When theavailable light drops below a minimum threshold (e.g., the device 100 isin a completely dark room with no windows as an extreme example),embodiments of the device 100 described below supply the ambient light(backlight) in order to facilitate gesture detection. Calibration of theALSs 110 based on the backlight is also detailed below.

FIG. 9 shows an arrangement of backlight sources 910 in an exemplarydevice 100 according to an embodiment of the invention. According to theembodiment shown in FIG. 9, the backlight sources 910 are under thescreen 120 and each is arranged in proximity to an ALS 110. That is,backlight source 910 a is proximate to ALS 110 x, backlight source 910 bis proximate to ALS 110 y, and backlight source 910 c is proximate toALS 110 z. Each backlight source 910 may be regarded as a sectionalflashlight that provides ambient light at one section of the device 100screen 120. A backlight source 910 may be on or off and additionally mayhave varying controllable levels of brightness according to differentembodiments. Adjustment of the backlight sources 910 through acalibration process is detailed below. When the ambient light in theenvironment in which the device 100 is being used (as indicated by thelight intensity 115 outputs of the ALSs 110) is above a threshold valuerepresenting the minimum acceptable ambient light level for gesturedetection, the backlight sources 910 may not be operational at all.Because a low light intensity 115 indication from only one or less thanall of the ALSs 110 is likely due to a localized issue (e.g., one of theALSs 110 is covered), the light intensity 115 sensed by most or all ofthe ALSs 110 is used to determine when the overall ambient light hasdropped below an acceptable level to facilitate gesture detection. Whenthe ambient light falls below this minimum threshold, one or more of thebacklight sources 910 may be turned on to maintain a range of operationof the ALSs 110 within limits that facilitate the gesture identifier 530being able to match the light intensity 115 levels with those in thegesture template 537. All of the backlight sources 910 a, 910 b, 910 cshown in FIG. 9 need not necessarily be turned on based on the sensedambient light being below the minimum threshold. Instead, only one ortwo of the backlight sources 910 a, 910 b, 910 c may be turned on. Also,in alternate embodiments to the one shown in FIG. 9, there may be asingle backlight source 910 or multiple backlight sources 910 may bearranged in a way that does not correspond with the arrangement of ALSs110. However, having each of the backlight sources 910 a, 910 b, 910 cproximate to each of the ALSs 110 x, 110 y, 110 z turned on facilitatesaccurately discerning direction of movement of an object 240. This isbecause having fewer than all of the backlight sources 910 a, 910 b, 910c on can be thought of as creating a bias. That is, if only backlightsources 910 a and 910 b were turned on based on insufficient ambientlight conditions, then ALS 110 z (because of backlight source 901 cbeing off) would sense lower light intensity 115 levels than ALSs 110 xand 110 y whether or not a gesture were being performed in front of ALS110 z. In addition, having fewer than all of the backlight sources 910a, 910 b, 910 c turned on in insufficient ambient lighting conditionsmay not bring the ALSs 110 x, 110 y, 110 z back to operation within anacceptable range.

When the ambient lighting conditions dictate that the backlight sources910 are needed, a calibration is performed for the ALSs 110. Anexemplary calibration is discussed with reference to Table 1 below.

TABLE 1 910a 910b 910c 110x 110y 110z 0 0 0  5 lux  5 lux  5 lux 1 0 018 lux 15 lux 15 lux 0 1 0 15 lux 18 lux 15 lux 0 0 1 15 lux 15 lux 18lux 1 1 1 26 lux 26 lux 26 luxTable 1 indicates an on (“1”) or off (“0”) state for each backlightsource 910 a, 910 b, 910 c shown in FIG. 9 along with the resultinglight intensity 115 (in photometric units of lux) sensed by each of theALSs 110 x, 110 y, 110 z. Thus, each row of the table represents a stateof the backlight sources 910 for which ambient light 115 values arerecorded. The backlight sources 910 are adjusted in intensity andduration to provide the light intensity 115 readings shown in Table 1.That is, through the adjustment of the backlight sources 910, the ALSs110 are calibrated to provide the exemplary light intensity 115 valuesshown in Table 1 regardless of the level of ambient light from sourcesother than the backlight sources 910 (when the ambient light level isbelow the minimum threshold). Thus, the calibration process provides therelative difference in light intensity of the backlight sources 910 a,910 b, 910 c needed and a time delay between each of the backlightsources 910 producing light and each of the ALSs 110 beginning sensingof light intensity 115. This information may also be used in thesynchronization between ALSs 110 and backlight sources 910 discussedbelow. The backlight sources 910 are monitored during their operation(e.g., at a rate of 10 times per second in an exemplary embodiment) toadjust the brightness of each backlight source 910 as needed.

To address potential false alarms, caused by a temporary obstruction tothe light source 210 for example, operation of the backlight sources 910may not begin until the light intensity 115 measurements of all the ALSs110 are below the minimum threshold value for a certain period of time.In this way, the backlight sources 910 are not likely to be activatedbased on either a momentary reduction in ambient light level below theminimum threshold or a gesture that includes blocking one or more of theALSs 110 from the light source 210. A user preference may be selected toprevent the backlight sources 910 from being operated, as well. Forexample, a user who leaves the device 100 on while sleeping in a darkroom will not be disturbed by the backlight sources 910 based onselecting a setting in the device 100. The backlight sources 910 mayoverride the existing screen 120 backlight of a device 100 thatfacilitates reading the device 120 screen in a dark environment. Thus,during a slide presentation in a dark conference room, for example, ifthe user wishes to use touch-less gestures to control the slidenavigation, he or she may disable the screen 120 backlight and enablethe backlight sources 910 based on settings selected in the device 100.When the backlight sources 910 are operational (have been turned on dueto ambient light level dropping below the minimum threshold) and theambient light level then exceeds another threshold representing amaximum acceptable ambient light level for gesture detection, one ormore of the backlight sources 910 may be turned off. This maximumthreshold value may have to be exceeded for a certain period of timebefore any of the backlight sources 910 is turned off to ensure that thebacklight sources 910 are not disabled based on a temporary flash. Thisperiod of time may be the same as or different from the period of timefor which the ambient light level must be below the minimum threshold toinitiate operation of the backlight sources 910. Based on the maximumthreshold, the backlight sources 910 are not left operational afterambient light sources (210) external to the device 100 have begun toprovide sufficient light for gesture detection. This may happen becausethe device 100 is moved to a more well-lit area or because a lightsource 210 is turned on or restored, for example. Phase synchronizationbetween the ALSs 110 and the backlight sources 910 (matching a frequencyof when the ALSs 110 sense light intensity 115 with a frequency of thebacklight sources 910) ensures sufficient sensitivity of the ALSs 110 todetect movements that make up gestures.

FIG. 10 is a block diagram of a system 1000 to control the backlightsources 910 according to an embodiment. The system 1000 includes aprocessor 1010 that receives information about the light intensity 115sensed by each of the ALSs 110. While the processor 1010 is shown asseparate from those discussed above as being part of the device 100, theprocessor 1010 may be one or more of the processors discussed above. Forexample, because the data collection engine 510 receives the lightintensity 115 measurements from the ALSs 110 and the system 610 receivesframes of data 520 collected by the data collection engine 510, theprocessor 1010 may be part of or work in conjunction with the gestureidentifier 530 processor or one or more processors of the system 610.The processor 1010 that controls the backlight sources 910 may alsoinclude other processors (e.g., DSP 801) of the device 100. Theprocessor 1010 sends one or more control signals 1020 to one or morebacklight sources 910 to affect their operation and to synchronize thebacklight sources 910 with the ALSs 110 as discussed above. Thesynchronization may require the processor 1010 to work in conjunctionwith the system 610 that controls the ALSs 110. The processor 1010controls the backlight sources 910 according to the minimum and maximumthresholds discussed above, and may operate the backlight sources 910based on the minimum or maximum threshold being crossed for a certainperiod of time.

FIG. 11 is a process flow of an exemplary method 1100 of performingtouch-less gesture detection under any ambient lighting conditions. Atblock 1110, monitoring the ambient light level may be done by theprocessor 1010. When the ambient light level is determined to be above aminimum threshold, performing gesture detection at block 1160 includesthe processes detailed above with reference to FIGS. 5 and 6. When theambient light level is determined to have fallen below a minimumthreshold, operating the backlight sources 910 at block 1120 includescalibrating the ALSs 110 (block 1130) as discussed with reference toTable 1 above and synchronizing the ALSs 110 and backlight sources 910(block 1140). Monitoring the backlight sources 910 at block 1150includes monitoring, reporting (e.g., to the processor 1010), andadjusting the level of brightness of each backlight source 910 asneeded. At any time (e.g., after performing gesture detection at block1160), monitoring the ambient light level at block 1110 may indicatethat the backlight sources 910 may be turned off. For example, themaximum threshold may be exceeded by the light intensity 115 outputs ofthe ALSs 110.

While several embodiments have been provided in the present disclosure,it should be understood that the disclosed systems and methods may beembodied in many other specific forms without departing from the spiritor scope of the present disclosure. The present examples are to beconsidered as illustrative and not restrictive, and the intention is notto be limited to the details given herein. For example, the variouselements or components may be combined or integrated in another systemor certain features may be omitted, or not implemented.

Also, techniques, systems, subsystems and methods described andillustrated in the various embodiments as discrete or separate may becombined or integrated with other systems, modules, techniques, ormethods without departing from the scope of the present disclosure.Other items shown or discussed as coupled or directly coupled orcommunicating with each other may be indirectly coupled or communicatingthrough some interface, device, or intermediate component, whetherelectrically, mechanically, or otherwise. Other examples of changes,substitutions, and alterations are ascertainable by one skilled in theart and could be made without departing from the spirit and scopedisclosed herein.

What is claimed is:
 1. A device to detect a gesture performed by anobject in touch-less communication with the device, the devicecomprising: two or more ambient light sensors arranged at respectivefirst surface locations of the device, each of the two or more ambientlight sensors configured to sense light intensity at the respectivefirst surface locations; one or more processors configured to controlone or more light sources at respective second surface locations of thedevice based on the light intensity sensed by the two or more ambientlight sensors, calibrate the two or more ambient light sensors based onthe one or more light sources, and detect the gesture based on the lightintensity sensed by each of the two or more ambient light sensors. 2.The device according to claim 1, wherein a number of the two or moreambient light sensors is equal to a number of the one or more lightsources and each respective first surface location corresponds with oneof the respective second surface locations.
 3. The device according toclaim 1, wherein the one or more processors monitors an ambient lightlevel based on the light intensity sensed by the two or more ambientlight sensors and initiates operation of the one or more light sourceswhen the ambient light level falls below a minimum threshold level for aperiod of time.
 4. The device according to claim 1, wherein the one ormore processors calibrate the two or more ambient light sensors byturning on each of the one or more light sources in turn to establish aplurality of states of the one or more light sources.
 5. The deviceaccording to claim 4, wherein the plurality of states includes all ofthe one or more light sources being turned off and all of the one ormore light sources being turned on.
 6. The device according to claim 4,wherein the one or more processors adjust the two or more ambient lightsensors to sense a specific light intensity at each of the plurality ofstates.
 7. The device according to claim 6, wherein the specific lightintensity for one of the two or more ambient light sensors is a samelight intensity as the specific light intensity for another of the twoor more ambient light sensors.
 8. The device according to claim 6,wherein the specific light intensity for one of the two or more ambientlight sensors is a different light intensity than the specific lightintensity for another of the two or more ambient light sensors.
 9. Amethod of detecting a gesture made by an object in touch-lesscommunication with a device, the method comprising: sensing, using twoor more ambient light sensors arranged at respective first surfacelocations of the device, light intensity at the respective first surfacelocations; controlling, using one or more processors, one or more lightsources at respective second surface locations based on the lightintensity sensed by the two or more ambient light sensors; calibrating,using the one or more processors, the two or more ambient light sensorsbased on the one or more light sources, and detecting, using the one ormore processors, the gesture based on the light intensity sensed by eachof the two or more ambient light sensors.
 10. The method according toclaim 9, further comprising the one or more processors monitoring anambient light level based on the light intensity sensed by the two ormore ambient light sensors and initiating operation of the one or morelight sources when the ambient light level falls below a minimumthreshold level for a period of time.
 11. The method according to claim9, wherein the calibrating includes the one or more processors turningon each of the one or more light sources in turn to establish aplurality of states of the one or more light sources.
 12. The methodaccording to claim 11, wherein the plurality of states include all ofthe one or more light sources being turned off and all of the one ormore light sources being turned on.
 13. The method according to claim11, further comprising the one or more processors adjusting the two ormore ambient light sensors to sense a specific light intensity at eachof the plurality of states.
 14. The method according to claim 13,wherein the specific light intensity for one of the two or more ambientlight sensors is a same light intensity as the specific light intensityfor another of the two or more ambient light sensors.
 15. The methodaccording to claim 13, wherein the specific light intensity for one ofthe two or more ambient light sensors is a different light intensitythan the specific light intensity for another of the two or more ambientlight sensors.